Defining test process management

Most activities result in one or more products, such as (master) test plan, reports, test scripts, test files, test logs, etc. Exceptions are supporting activities, which usually do not deliver any tangible products. A choice has to be made as to whether to register the progress at the level of activities or at the level of products, with the further possibility of the mix form. The advantage of managing at the level of products is that these are easier to measure than activities: it is easier to judge whether a product is 80% ready than an activity, and more and more development and project management methods manage on the basis of products. With the identification of activities or products, attention must be paid to the required degree of detail. Is it important to register an activity of several hours separately, or is it more efficient to register this as a part of a bigger activity? This is determined in consultation with the client.

Defining infrastructure management

As regards the distribution of the management tasks, the various aspects of the test infrastructure can be divided into two groups:

Technical management

• test environment (hardware and software; management procedures)
• test files (physical)
• networks for test environment and office setup
• technical office setup
• test tools

The most important tasks are:

• Version management
• Configuration management
• Solving problem areas
• Making logging available
• Backup & restore
• Recovery
• (Technical) monitoring
• Issuing authorisations
• Providing availability
• Implementing changes
• Maintenance
• Dealing with breakdowns.

The technical management tasks that have to be carried out belong to the role of test infrastructure coordinator. With the execution of these tasks, support is given as required by the supplier or a department, such as system management or infrastructure services.

Logistical management

This is the non-technical part of the office setup, such as canteen provisions, transport, entry passes, etc. The tasks in the context of logistical management are not test-specific and as such are not discussed further in this method.

Defining test product management

Below is a kick-start to a test-product management procedure. The procedure consists of four steps:

Delivery

The products to be managed are delivered by the testers to the manager. Preferably, the delivered files are placed in a separate directory. The products should be delivered complete (among other things, supplied with a version date and version number). The manager checks for completeness. The following are some of the items that could be checked:

• Name of author
• Type of document (also in document name)
• The definitive version number and version date
• Accuracy of references to other documentation (the test products should refer clearly to the associated test object and test basis)
• Mutations overview: overview of the versions, version dates and reason for change, including the name of the person who made the change
• Products in electronic form should be delivered with a fixed nomenclature, in a form that includes the version number.

Registration

The manager registers the delivered products in his administration on the basis of supplier’s name, name of product, date and version number. At the same time, it is registered how long the relevant products should be kept. In certain cases, it may also be necessary to include the information on products related to the product to be registered.

We find this in organisations where traceability is an important issue, for example because of legal obligations. With the registration of changed products, the manager should ensure that the consistency between the various products is preserved.

Archiving

A distinction is made between new and changed products. Stated roughly, new products are added to the archive and changed products replace the previous
version.

Consultation

The issue of products to project team members or third parties takes place by means of a copy of the requested products. The manager registers which version of the products has been issued to whom and when.

Traceability

Partly because of legislation (IFRS, SOX, FDA (Food and Drug Administration) and FAA (Federal Aviation Administration)), it is becoming increasingly important
to demonstrate both that testing is being carried out and also what exactly is being tested. Showing what is being tested is achieved through traceability (demonstrating which test cases bear a relation to which part of the test basis). The proof that testing is actually being performed has to be supplied through explicit reporting. A subsequent requirement is to provide proof that the defects have been dealt with. If these stringent requirements concerning traceability and submission of proof are to be met, then the test product management, defects management and quality assurance in respect of testing should be tailored to this end (and extra budget made available for it!). The test management should be set up in such a way that the traceability and evidence can be followed step by step. This means that:

• It is clearly indicated in the test specifications from which part of the test basis these are derived
• With the test execution, the evidence to be submitted relates to which test cases have actually been executed
• It is made apparent which test cases have led to which defects
• The evidence to be submitted is established during the retest; which defects have been solved and approved in a retest.

This apart, traceability has the following big advantages for testing:

• Much insight is gained into the quality and depth of the test, because from the requirements, the functional and the technical design and the software, it is known with which test cases these have been checked (or will be). The chances of omissions in the test are therefore much reduced.
• With changes in the test basis or the test object, it can be quickly deduced which test cases need to be amended and/or carried out anew.
• If, owing to pressures of time, it is not possible to carry out all the planned tests, test cases will have to be scrapped. Because the relationship with requirements, specifi cations and software is known, we can scrap those test cases of which the associated requirement or specification presents the least risk in production and it is clear with which requirements or specifications no, or no well-founded, decision is possible on the quality.

If we want to set up the test to provide traceability, then the deployment of tools for test product management is more or less indispensable.